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Video and/or Audio Processing 

CROSS-REFERENCE TO RELATED APPLICATION 

This application is related to an application entitled "Stereoscopic Video", 
to inventor Thomas Algie Abrams, Jr, assigned to Microsoft Corporation, filed 

concurrently on January 28, 2002 and having Serial No. _/ and attorney 

Docket No. MS1-891US, tiie contents of which are incorporated by reference 
herein. 

TECHNICAL FIELD 

This invention relates generally to methods, devices, systems and/or storage 
media for video and/or audio processing. 

BACKGROUND 

In a typical digital video process, digital content is created through use of a 
computer, a digital camera, and/or by converting existing analog content, such as 
35mm film, into a digital format. In general, a downward progression exists 
wherein the resolution, and hence quality, of content distributed to a viewer is 
much less than that of the original content. For example, a professional digital 
video camera may acquire image data at a resolution of 1280 pixel by 720 lines, a 
frame rate of 24 frames per second (fps) and a color depth of 24 bits. The 
acquisition rate for such content is approximately 530 million bits per second 
(Mbps); thus, two hours of filming corresponds to almost 4 trillion bits of data 
(Tb). For viewing, this content must be distributed at approximately 530 Mbps or 
downloaded as a file having a size of approximately 4 Tb. At present, bandwidths 
and recording media commonly used for commercial distribution of digital content 
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cannot handle such requirements. Thus, re-sampling and/or compression need to 
be applied to reduce the bit rate and/or file size. 

Perhaps the most widely used method of compression is specified in tiie 
MPEG-2 standard. Products such as digital television (DTV) set top boxes and 
DVDs are based on the MPEG-2 standard. As an example, consider a DVD player 
with a single sided DVD disk that can store approximately 38 Gb. To fit the 
aforementioned 2 hours of video onto this disk, consider first, a re-sampling 
process that downgrades the video quality to a format having a resolution of 720 
pixel by 486 line, a frame rate of approximately 24 fps and a color depth of 16 
bits. Now, mstead of a bit rate of 530 Mbps and a file size of 4 Tb, the content has 
a bit rate of approximately 130 Mbps and a file size of approximately 1 Tb. To fit 
the 1 Tb of content on a 38 Gb single sided DVD disk, a compression ratio of 
approximately 30:1 is required. When storage of audio and sub-tities is desired, an 
even higher compression ratio, for example, of approximately 40:1, is required. In 
addition, to decode and playback the 38 Gb of compressed content in 2 hours, an 
average bit rate of approxunately 5 Mbps is required. 

In general, MPEG-2 compression ratios are typically confined to 
somewhere between approximately 8:1 and approxunately 30:1, which some have 
referred to as the MPEG-2 compression "sweet spot". Further, with MPEG-2, 
transparency (i.e., no noticeable discrepancies between source video and 
reconstructed video) occurs only for conservative compression ratios, for example, 
between approximately 8:1 and approximately 12:1. Of course, such conservative 
compression ratios are inadequate to allow for storage of the aforementioned 130 
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Mbps, 2 hour video on a DVD disk. Thus, to achieve a high degree of 
transparency, source content is often pre-processed (e.g., re-sampled) prior to 
MPEG-2 compression or lower resolution source content is used, for example, 352 
pixel by 480 lines at a frame rate of 24 ^s and a color depth of 16 bits. Two hours 
of such lower resolution content requires a compression ratio of approximately 
12:1 to fit a single sided DVD disk. 

In practice, however, for a variety of reasons, MPEG-2 compression ratios 
are typically around 30:1. For example, a reported MPEG-2 rate-based "sweet 
spot" specifies a bit rate of 2 Mbps for 352 pixel by 480 line and 24 ^s content, 
which reportedly produces an almost NTSC broadcast quality result that is also a 
"good" substitute for VHS. To achieve a 2 Mbps rate for the 352 pixel by 480 line 
and 24 fps content requires a compression ratio of approximately 30:1, which 
again, is outside the conservative compression range. Thus, most commercial 
applications that rely on MPEG-2 for video, have some degree of quality 
degradation and/or quality limitations. 

One way to increase video quality involves maintaining a higher resolution 
(e.g., maintaining more pixels). Another way to increase video quality involves 
use of better compression algorithms, for example, algorithms that maintain 
subjective transparency for compression ratios greater than approximately 12:1 
and/or achieve VHS quality at compression ratios greater than 30:1. Of course, a 
combination of both higher resolution and better compression algorithms can be 
expected to produce the greatest increase in video quality. For example, it would 
be desirable to maintain the 1280 pixel by 720 line resolution of the 
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aforementioned digital video and it would also be desirable to fit such content onto 
a single sided DVD disk or other DVD disk. In addition, it would be desirable to 
transmit such content in a data stream. Technologies for accompHshing such 
tasks, as well as other tasks, are presented below. 



SUMMARY 

Various technologies are described herein that pertain generally to digital 
video. Many of these technologies can lessen and/or eliminate the need for a 
downward progression in video quality. Other technologies allow for new 
manners of distribution and/or display of digital video. In general, various 
technologies described herein allow for compression, storage, transmission and/or 
display of video having a resolution of, for example, greater than approximately 
720 pixel by approximately 576 line. In addition, various technologies described 
herein can provide DVD quality. An exemplary method includes receiving and/or 
requesting digital video data, compressing the digital video data and transmitting 
and/or storing the compressed digital video data while another exemplary method 
includes receivmg and/or requesting compressed digital video data, decompressing 
the digital video data and displaying the decompressed digital video data. Yet 
other method, devices, systems and/or storage media are further described herein. 

Additional features and advantages of the various exemplary methods, 
devices, systems, and/or storage media will be made apparent from the following 
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detailed description of illustrative embodiments, which proceeds with reference to 
the accompanying figures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

A more complete understanding of the various methods and arrangements 
described herein, and equivalents thereof, may be had by reference to the 
following detailed description when taken in conjunction with the accompanying 
drawings wherein: 

Fig. 1 is a block diagram generally illustrating an exemplary computer 
system on which the exemplary methods and exemplary systems described herein 
may be implemented. 

Fig. 2 is a block diagram illustrating an exemplary method for converting 
film images to streamable and/or storable digital data optionally suitable for 
transmission to a display device. 

Fig. 3 is a block diagram illustrating an exemplary method for converting 
information to a particular format using video and/or audio codecs. 

Fig. 4 is a block diagram illustrating an exemplary process for compression 
and decompression of image data. 

Fig. 5 is a block diagram illustrating an exemplary method for producing a 
stream and/or file. 
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Fig. 6 is a block diagram illustrating an exemplary device and/or system for 
digital storage and/or structuring. 

Fig. 7 is a block diagram illustrating an exemplary method for processing 
video data. 

Fig. 8 is a block diagram illustrating an exemplary method for processing 
video data. 

Fig. 9 is a block diagram illustrating an exemplary method for delivering a 
stream. 

Fig. 10 is a block diagram illustrating an exemplary method for displaying 
video and/or audio data from an I/O device. 

Fig. 11 is a block diagram illustrating an exemplary method for displaymg 
video and/or audio data from a computer. 

Fig. 12 is a block diagram illustratmg an exemplary method for processing 
video data suitable for display on a display device having a lenticular lens or the 
like. 

Fig. 13 is a block diagram illustrating an exemplary method for displaying 
video from a decoded stream and/or file. 
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Fig. 14 is a graph of video data rate in Gbps versus processor speed in GHz 
for a computer having a single processor. 

DETAILED DESCRIPTION 

Turning to the drawings, wherein like reference numerals refer to like 
elements, various methods are illuslrated as being unplemented in a suitable 
computing environment. Although not required, the methods will be described in 
the general context of computer-executable instructions, such as program modules, 
being executed by a personal computer. Generally, program modules include 
routines, programs, objects, components, data structures, etc. that perform 
particular tasks or implement particular abstract data types. Moreover, those 
skilled LQ the art will appreciate that the methods and converters may be practiced 
with other computer system configurations, including hand-held devices, multi- 
processor systems, microprocessor based or programmable consumer electronics, 
network PCs, minicomputers, mainframe computers, and tiie like. The methods 
may also be practiced in distributed computing environments where tasks are 
performed by remote processmg devices that are linked through a conraiunications 
network. In a distributed computing environment, program modules may be 
located in both local and remote memory storage devices. 

In some diagrams herein, various algorithmic acts are summarized in 
individual "blocks". Such blocks describe specific actions or decisions that are 
made or carried out as the process proceeds. Where a microcontroller (or 
equivalent) is employed, the flow charts presented herein provide a basis for a 
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"control program" or software/firmware that may be used by such a 
microcontroller (or equivalent) to effectuate the desired control of the stimulation 
device. As such, the processes are implemented as machine-readable instructions 
stored in memory that, when executed by a processor, perform the various acts 
illustrated as blocks. 

Those skilled in the art may readily write such a control program based on 
the flow charts and other descriptions presented herein. It is to be understood and 
appreciated that the inventive subject matter described herein includes not only 
stimulation devices when programmed to perform the acts described below, but 
the software that is configured to program the microcontrollers and, additionally, 
any and all computer-readable media on which such software might be embodied. 
Examples of such computer-readable media include, without limitation, floppy 
disks, hard disks, CDs, RAM, ROM, flash memory and the like. 

Fig.l illustrates an example of a suitable computing environment 120 on 
which the subsequently described exemplary methods may be implemented. 
Exemplary computing environment 120 is only one example of a suitable 
computing environment and is not intended to suggest any limitation as to the 
scope of use or functionality of the improved methods and arrangements described 
herein. Neither should computing environment 120 be interpreted as having any 
dependency or requirement relating to any one or combination of components 
illustrated in computing environment 120. 
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The methods and arrangements herein are operational with numerous other 
general purpose or special purpose computing system environments or 
configurations. Examples of well known computing systems, environments, 
and/or configurations that may be suitable include, but are not Ihnited to, personal 
computers, server computers, thin clients, thick clients, hand-held or laptop 
devices, multiprocessor systems, microprocessor-based systems, set top boxes, 
programmable consumer electronics, network PCs, minicomputers, mainfi-ame 
computers, distributed computing environments that include any of the above 
systems or devices, and the like. 

As shown in Fig. 1, computing environment 120 includes a general-purpose 
computing device in the form of a computer 130. The components of computer 
130 may include one or more processors or processing units 132, a system 
memory 134, and a bus 136 that couples various system components including 
system memory 134 to processor 132. 

Bus 136 represents one or more of any of several types of bus structures, 
including a memory bus or memory controller, a peripheral bus, an accelerated 
graphics port, and a processor or local bus using any of a variety of bus 
architectures. By way of example, and not limitation, such architectures include 
Industry Standard Architecture (ISA) bus. Micro Channel Architecture (MCA) 
bus, Enhanced ISA (EISA) bus. Video Electronics Standards Association (VESA) 
local bus, and Peripheral Component Interconnects (PCI) bus also known as 
Mezzanine bus. 
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Computer 130 typically includes a variety of computer readable media. 
Such media may be any available media that is accessible by computer 130, and it 
includes both volatile and non-volatile media, removable and non-removable 
media. 

In Fig. 1, system memory 134 includes computer readable media in the 
form of volatile memory, such as random access memory (RAM) 140, and/or non- 
volatile memory, such as read only memory (ROM) 138. A basic input/output 
system (BIOS) 142, containing the basic routines that help to transfer information 
between elements within computer 130, such as durmg start-up, is stored in ROM 
138. RAM 140 typically contains data and/or program modules that are 
immediately accessible to and/or presently being operated on by processor 132. 

Computer 130 may further include other removable/non-removable, 
volatile/non-volatile computer storage media. For example. Fig. 1 illustrates a 
hard disk drive 144 for reading from and writing to a non-removable, non-volatile 
magnetic media (not shown and typically called a "hard drive"), a magnetic disk 
drive 146 for reading from and writing to a removable, non-volatile magnetic disk 
148 (e.g., a "floppy disk"), and an optical disk drive 150 for reading from or 
writing to a removable, non- volatile optical disk 152 such as a CD-ROM, CD-R, 
CD-RW, DVD-ROM, DVD-RAM or other optical media. Hard disk drive 144, 
magnetic disk drive 146 and optical disk drive 150 are each connected to bus 136 
by one or more interfaces 154. 
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The drives and associated computer-readable media provide nonvolatile 
storage of computer readable instructions, data structures, program modules, and 
other data for computer 130. Although the exemplary environment described 
herein employs a hard disk, a removable magnetic disk 148 and a removable 
optical disk 152, it should be appreciated by those skilled in the art that other types 
of computer readable media which can store data that is accessible by a computer, 
such as magnetic cassettes, flash memory cards, digital video disks, random access 
memories (RAMs), read only memories (ROM), and the like, may also be used in 
the exemplary operating environment. 

A number of program modules may be stored on the hard disk, magnetic 
disk 148, optical disk 152, ROM 138, or RAM 140, including, e.g., an operating 
system 158, one or more application programs 160, other program modules 162, 
and program data 164. 

The improved methods and arrangements described herein may be 
implemented v^ithin operating system 158, one or more application programs 160, 
other program modules 162, and/or program data 164. 

A user may provide commands and information into computer 130 through 
input devices such as keyboard 166 and pointing device 168 (such as a "mouse"). 
Other input devices (not shown) may include a microphone, joystick, game pad, 
satellite dish, serial port, scanner, camera, etc. These and other input devices are 
connected to the processing unit 132 through a user input interface 170 that is 
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coupled to bus 136, but may be connected by other interface and bus structures, 
such as a parallel port, game port, or a universal serial bus (USB). 

A monitor 172 or other type of display device is also connected to bus 136 
via an interface, such as a video adapter 174. In addition to monitor 172, personal 
computers typically include other peripheral output devices (not shown), such as 
speakers and printers, which may be connected through output peripheral interface 
175. 

Logical connections shown in Fig. 1 are a local area network (LAN) 177 
and a general wide area network (WAN) 179. Such networking environments are 
commonplace in offices, enterprise-wide computer networks, intranets, and the 
Internet. 

When used in a LAN networking environment, computer 130 is connected 
to LAN 177 via network interface or adapter 186. When used in a WAN 
networking environment, the computer typically includes a modem 178 or other 
means for establishing communications over WAN 179. Modem 178, which may 
be internal or external, may be connected to system bus 136 via the user input 
interface 170 or other appropriate mechanism. 

Depicted m Fig. 1, is a specific implementation of a WAN via the Internet. 
Here, computer 130 employs modem 178 to estabUsh communications with at 
least one remote computer 182 via the Internet 180. 
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In a networked environment, program modules depicted relative to 
computer 130, or portions thereof, may be stored in a remote memory storage 
device. Thus, e.g., as depicted in Fig. 1, remote application programs 189 may 
reside on a memory device of remote computer 182. It will be appreciated that the 
network connections shown and described are exemplary and other means of 
establishing a communications link between the computers may be used. 

Overview 

Various technologies are described herein that pertain generally to digital 
video. Many of these technologies can lessen and/or eliminate the need for a 
downward progression in video quality. Other technologies allow for new 
manners of distribution and/or display of digital video. As discussed in further 
detail below, such technologies include, but are not hmited to: exemplary methods 
for producing a digital video stream and/or a digital video file; exemplary methods 
for producing a transportable storage medium containing digital video; exemplary 
methods for displaying digital video; exemplary devices and/or systems for 
producing a digital video stream and/or a digital video file; exemplary devices 
and/or systems for storing digital video on a transportable storage medium; 
exemplary devices and/or systems for displaying digital video; and exemplary 
storage media for storing digital video. 

Various exemplary methods, devices, systems, and/or storage media are 
described with reference to fi:ont-end, intermediate, back-end, and/or front-to-back 
processes and/or systems. While specific examples of commercially available 
hardware, software and/or media are often given throughout the description below 
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in presenting front-end, intermediate, back-end and/or front-to-back processes 

and/or systems, the exemplary methods, devices, systems and/or storage media, 
are not limited to such commercially available items. 

Description of Exemplarv Methods. Devices. Systems, and/or Media 

Referring to Fig. 2, a block diagram of an exemplary method for producing 
a digital video stream 200 is shov^n. In a shooting block 210, a cinematographer 
uses a camera to fihn, or capture, images, or video, on photographic film. In 
general, the photographic film has an industry standard format, e.g., 70 mm, 35 
mm, 16 mm, or 8 mm, Sound, or audio, recorded as an analog track and/or as a 
digital track on magnetic recording media and/or optical recording media, may 
also accompany the video. A photographic film may include magnetic recording 
media and optical recording media for audio recording. Common audio formats 
for film include, but are not limited to, 6 track/channel DOLBY DIGITAL® 
format (Dolby Laboratories Licensing Corporation, San Francisco, California) and 
8 track/channel SDDS SONY DYNAMIC DIGITAL SOUND® format (Sony 
Corporation, Tokyo, Japan). In addition, a 6 track/channel DTS® format (Digital 
Theatre Systems, Inc., Westlake Village, Califomia), a CD-based format, may also 
accompany a film. Of course, other CD-based systems may be used. Editing 
and/or rerecording optionally occur after filming to produce a final film and/or a 
near final film having analog video and optionally digital audio and/or analog 
audio. 

As shown in Fig. 2, in a fihn transfer block 220, the film is transferred to a 
telecine. However, in an altemative, a digital camera is used, e.g., to optionally 
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alleviate the need for analog film. A variety of digital cameras are commercially 
available, such as, but not limited to, a SONY® digital camera (Sony Corporation, 
Japan). Use of a digital camera can alleviate the need for an analog-to-digital 
conversion block or it can substitute as an analog-to-digital conversion block. 
Exemplary SONY® digital cameras include, but are not limited to, the SONY® 
HDW-F900 and HDW-700A digital cameras. The SONY® HDW-F900 digital 
camera features HAD CCD technology, which combines a 3-CCD HD color 
digital camera, a 12-bit A/D converter with advanced digital signal processing to 
deliver image resolution up to 1,920 pixels by 1,080 line. The SONY® HDW- 
700A digital camera is a 1080i (1080 line interlace) compliant 2 million-pixel 
RGB camera utilizing 10-bit digital signal processing. In addition, SONY® 
HDCAM equipment is optionally used for recording and/or processing (see blocks 
described below). Such equipment includes, but is not limited to, the SONY® 
HDW-F500 HDCAM editing VTR. According to various exemplary methods, 
etc., disclosed herein, any digital camera capable of acquiring video with a pixel 
and/or a line resolution of at least approximately 720 is suitable. 

In an analog-to-digital conversion block 230, a telecine (or equivalent 
device) converts analog video to digital video. Commercially available telecines 
include CCD telecines and CRT telecines and both types are suitable for the 
analog-to-digital conversion block 230. Telecines capable of digital resolution in 
excess of 1920 pixels per line and/or 1080 lines are also suitable for use with 
various exemplary methods, devices, systems and/or storage media described 
herein. 
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Regarding digital video formats. Table 1, below, presents several commonly 
used digital video formats, including 1080 x 1920, 720 x 1280, 480 x 704, and 480 
X 640, given as number of lines by number of pixels. 



Table 1. Common Digital Video Formats 









Frame Rate 


Seouence 
p or i 


1080 


1920 


16:9 


24,30 


progressive 


1080 


1920 


16:9 


30, 60 


interlaced 


720 


1280 


16:9 


24, 30, 60 


progressive 


480 


704 


4:3 or 16:9 


24, 30, 60 


progressive 


480 


704 


4:3 or 16:9 


30 


interlaced 


480 


640 


4:3 


24, 30, 60 


progressive 


480 


640 


4:3 


30 


interlaced 



Regarding high definition television (HDTV), formats generally include 
1,125 line, 1,080 line and 1,035 line interlace and 720 line and 1,080 line 
progressive formats in a 16:9 aspect ratio. According to some, a format is high 
definition if it has at least twice the horizontal and vertical resolution of the 
standard signal being used. There is a debate as to whether 480 line progressive is 
also "high definition''; it provides better resolution than 480 line interlace, making 
it at least an enhanced definition format. Various exemplary methods, devices, 
systems and/or storage media presented herein cover such formats and/or other 
formats. 
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In the analog-to-digital conversion block 230, the conversion device (e.g., 
telecine) outputs digital data in a suitable digital format, optionally according to a 
suitable standard for digital data transmission. While a variety of transmission 
standards exist, an exemplary suitable standard for digital data transmission is the 
Society of Motion Picture and Television Engineers (SMPTE) 292 standard ("Bit- 
Serial Digital Interface for High-Definition Television Systems"), which is 
typically associated with high definition systems (e.g., HDTV). In particular, the 
serial digital interface standard, SMPTE 292M, defines a universal medium of 
interchange for uncompressed digital data between various types of video 
equipment (camera's, encoders, VTRs, ...) at data rates of approximately 1.5 Gbps. 
Another exemplary suitable standard is the SMPTE 259M standard ("10-Bit 4:2:2 
Component and 4fsc Composite Digital Signals - Serial Digital Interface")^ which 
is typically associated with standard definition systems (e.g., SDTV). The SMPTE 
259M standard includes a data transmission rate of approximately 0.27 Gbps. 
Suitable source formats for use with the SMPTE serial digital interface standards 
may include, but are not limited to, SMPTE 260M, 295M, 274M and 296M. Such 
formats may include a 10-bit YCbCr color space specification and a 4:2:2 
sampling format and/or other color space specifications and/or sampling formats 
such as, for example, those described below. The various exemplary methods, 
devices, systems and/or storage media disclosed herein and equivalents thereof are 
not limited to the specifically mentioned SMPTE standards as other standards 
exist and/or are being created by organization such as the SMPTE. In addition, 
use of a non-standard transmission specification is also possible. 
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In general, digital video data typically has an 8-bit word and/or 10-bit word 
(also know as bits per sample) and a color space specification usually having an 
associated sampling format; this often results in an overall bits per pixel (or bit 
depth) of, for example, approximately 8, 16, 20, 24, 30 and 32. Of course, other 
word sizes and bit depths may exist and be suitable for use with various exemplary 
methods, devices, systems and/or storage media described herein. A variety of 
color space specifications also exist, including RGB, "Y, B-Y, R-Y", YUV, YPbPr 
and YCbCr. These are typically divided into analog and digital specifications, for 
example, YCbCr is associated with digital specifications (e.g., CCIR 601 and 656) 
while YPbPr is associated with analog specifications (e.g., EIA-770.2-a, CCIR 
709, SMPTE 240M, etc.). The YCbCr color space specification has been 
described generally as a digitized version of the analog YUV ^d YPbPr color 
space specifications; however, others note that CbCr is distinguished fi-om PbPr 
because in the latter the luma and chroma excursions are identical while in the 
former they are not. The CCIR 601 recommendation specifies an YCbCr color 
space with a 4:2:2 sampling format for two-to-one horizontal subsampling of Cb 
and Cr, to achieve approximately 2/3 the data rate of a typical RGB color space 
specification. In addition, the CCIR 601 recommendation also specifies that: 4:2:2 
means 2:1 horizontal downsampling, no vertical downsampling (4 Y samples for 
every 2 Cb and 2 Cr samples in a scanline); 4:1:1 typically means 4:1 horizontal 
downsampling, no vertical downsampling (4 Y samples for every 1 Cb and 1 Cr 
samples in a scanline); and 4:2:0 means 2:1 horizontal and 2:1 vertical 
downsampling (4 Y samples for every Cb and Cr samples in a scanline.). The 
CCIR 709 recommendation includes an YPbPr color space for analog HDTV 
signals while the YUV color space specification is typically used as a scaled color 
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Space in composite NTSC, PAL or S-Video. Overall, color spaces such as YPbPr, 
YCbCr, Photo YCC and YUV are mostly scaled versions of "Y, B-Y, R-Y" that 
place extrema of color difference channels at more convenient values. As an 
example, the digital data output from the analog-to-digital conversion block 230 
optionally includes a 1080 line resolution format, a YCbCr color space 
specification, and is transmittable according to the SMPTE 292M standard. Of 
course, a variety of other resolution formats, color space specifications and/or 
transmission standards may be used. In general, a resolution, a frame rate, and a 
color space specification together with a sampling format will determine an 
overall bit rate. 

Table 2 below lists a variety of video standards and associated bit rates. 



Table 2. Exemplary video formats and associated information. 



Format 


Pixels/line 


Lines/frame 


Pixels/frame 




Mps 


Bits/pixel 


Approx. 

Gbps 


SVGA 


800 


600 


480,000 


72 


34.6 


8 


0.27 


NTSC 


640 


480 


307,200 


30 


9.2 


24 


0.22 


PAL 


580 


575 


333,500 


50 


16.7 


24 


0.40 


SECAM 


580 


575 


333,500 


50 


16.7 


24 


0.40 


HDTV 


1920 


1080 


2,073,600 


30 


62.2 


24 


1.5 


Film* 


2000 


1700 


3,400,000 


24 


81.6 


32 


2.6 



♦Exemplary non-limiting fihn. 



Another exemplary video standard not included in Table 2 is for video 
having a resolution of 1920 pixel by 1080 line, a frame rate of 24 f^s, a 10-bit 
word and RGB color space with 4:2:2 sampling. Such video has on average 30 
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bits per pixel and an overall bit rate of approximately L5 Gbps. Yet another 
exemplary video standard not included in Table 2 is for video having a resolution 
of 1280 pixel by 720 line, a frame rate of 24 fps, a 10-bit word and a YCbCr color 
space with 4:2:2 sampling. Such video has on average 20 bits per pixel and an 
overall bit rate of approximately 0.44 Gbps. Note that a technique (known as 3:2 
pulldown) may be used to convert 24 frames per second film to 30 frames per 
second video. According to this technique, every other film frame is held for 3 
video fields resulting in a sequence of 3 fields, 2 fields, 3 fields, 2 fields, etc. Such 
a technique is optionally used in the analog-to-digital conversion block 230 or 
other blocks. 

As shown in Fig. 2, digital data output from the analog-to-digital 
conversion block 230 are input to a digital recording block 240. According to the 
exemplary method 200, the digital recording block 240, while shown in Fig. 2, is 
optional. Alternatively, the digital data ouput from the analog-to-digital 
conversion block 230 are input directly to a computer or device (e.g., see device 
610 of Fig. 6). In general, such a computer or device also includes storage 
capabilities. Referring again to Fig. 2, in the digital recording block 240, a 
recorder records digital data that includes video data, and optionally audio data, to 
a recording medium or media. For example, suitable recorders include, but are not 
limited to, tape-based and/or disk-based recorders. Exemplary non-limiting tape- 
based recorders include the Panasonic AJ-HD3700 D-5 HD multi-format 
recording system and the Philips DCR 6024 HDTV Digital Video Tape Recorder 
(also known as the Voodoo Media Recorder). Both of these commercially 
available recorders accept digital serial input according to the SMPTE 259M 
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and/or SMPTE 292M transmission standards. Further, both recorders can preserve 
1920 pixel X 1080 line resolution. 

The Panasonic AJ-HD3700 D-5 HD is a mastering-quality DTV/HDTV 
videotape recorder capable of performing mastering, high-definition cinema, 
television commercial and multi-format DTV and HDTV program production 
tasks. The AJ-HD3700 recorder can support standard definition and multiple 
high-definition video formats without hardware or software exchange, play back 
existing 525 line standard D-5 or D-5 HD cassettes and can record 10-bit 
uncompressed 480/601 standard-definition video with pre-read, in addition to 
1080/24p/25p, 1080/601, 1080/501, and 720/60p high-definition standards. In 
addition the recorder can slew between 24 and 25Hz frame rates for intemational 
(PAL) program duplication from a 1080/24p master. Both analog audio I/O and 
metadata recording and playback are supported as standard features. The D-5 
standard is a 10-bit 4:2:2 non-compressed component digital video recorder and 
suitable for high-end post production as well as more general studio use. The D-5 
HD standard (or HD D5 standard) provides for use of a compression algorithm to 
achieve about 4:1 lossless compression which may be suitable or acceptable for 
HDTV recordings. 

The Philips Voodoo recorder can record a variety of formats, including 
HDTV (or DTV) 4:2:2 YCrCb sampled formats (e.g., 1920 pixels x 1080 lines 
from 24p to 60i) without using any compression (24p is 24 fps progressive while 
601 is 60 ^s interlaced). The Philips Voodoo recorder is primarily based on the 
D6 recording format, which is a digital tape format that uses a 19 mm helical-scan 
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cassette tape to record uncompressed high definition television material at 1.88 
Gbps. The D6 standard includes SMPTE 277M and 278M standards and accepts 
both the European 1250/50 interlaced format and the Japanese 260M version of 
the 1125/60 interlaced format which uses 1035 active lines. 

Other suitable devices suitable for use in the recording block 240 are 
marketed and/or sold under the mark ACCOM® (Accom, Inc., Menlo Park, 
California). For example, the ACCOM® WSD®/HD device can record high 
definition and/or standard definition video on to storage disks (e.g., using SCSI 
disk drives). Such devices are sometimes referred to as digital disk recorder 
(DDR) devices; thus, some DDR devices may be suitable for use as a recorder. 
The AcCOM® WSD®/HD device can record uncompressed high definition video 
using a 10-bit 4:2:2 color format; it supports full 10-bit uncompressed I/O and 
storage of ITU-R BT.601-4 (CCIR 601) standard definition formats aad 720 line 
and 1080 line high definition formats. The ACCOM® WSD®/HD device can also 
use WINDOWS® file systems (e.g., NT® file system, 2000® file system, etc.) 
and/or the QUICKTIME® file format (Apple Computer, Inc., Cupertino, 
Califomia) for storage of video data. The ACCOM® WSD®/HD device 
optionally uses the QUICKTIME® file format as a native format for data storage. 
The QUICKTIME® file format includes two beisic structures for storing 
information: classic atoms and QT atoms. Both classic atoms, which are simple 
atoms, and QT atoms, which are atom container atoms, allow for construction of 
arbitrarily complex hierarchical data structures. Atoms consist of a header, 
followed by atom data. An atom's header contains the atom's size and type fields, 
giving the size of the atom in bytes and its type. Because of the limitations of the 
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classic atom structure, which require knowledge of offsets in to move through the 
atom tree, QT atoms are used which have an enhanced data structure that provide a 
more general-purpose storage format and remove some of the ambiguities that 
arise when using simple atoms. The QUICKTIME® file format supports storage 
of uncompressed (e.g., YCbCr or "YUV" 4:2:2, RGB, etc.) and compressed 
(JPEQ MPEQ etc.) video data. Of course, the recording block 240 is not limited 
to recorders that store data in a QUICKTIME® format. Another suitable, but non- 
Ihniting format is the WINDOWS MEDIA™ format, in addition, other formats 
may be suitable. Further, a recorder optionally compresses data using lossy and/or 
lossless compression. 

As with the aforementioned exemplary non-limiting recorders, the 
ACCOM® WSD®/HD device can input and/or output digital video using a serial 
digital interface according to SMPTE standards (e.g., 259 M, 292M). For 
example, using the SMPTE 292M specification, the ACCOM® WSD®/HD device 
can input and/or output 10-bit high definition video at approximately 1.5 Gbps. 
The ACCOM® WSD®/HD device also has audio storage options wherein various 
formats support both video and audio. Disk-based storage options include Med^a 
Corporation (Westlake Village, California) 78 gigabyte (GB) VideoRAID/RT, e.g., 
for standard definition storage, and a plurality of VideoRAID/RTs, e.g., for high 
definition storage, wherein capacities can range firom approxunately 78 GB to over 
10 terabyte (TB). As discussed in the background section, the 1280 pixel by 720 
Ime 2 hour video required a file size of approximately 4 Tb, which is 
approximately 0.5 TB; hence recorders, whether tape-based and/or disk-based, 
should have sufficient storage capabilities. The ACCOM® WSD®/HD device 
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supports gigabit Ethernet and/or WINDOWS® networking (e.g., WINDOWS® 
2000® networking). According to the exemplary method 200, a recorder, which is 
optional, optionally includes a network interface, such as, Ethernet, WINDOWS® 
and/or other interface. 

Yet other exemplary, non-limiting devices suitable for use in the digital 
recording block 240 include devices manufactured and/or sold by Post 
Impressions, Inc. (Culver City, California) under the mark "spiRINT". The 
spiRINT diskstation device includes SDRAM (e.g., 1 GB), an input for SMPTE 
292 transmission video, and arrays of storage disks (e.g., 3.2 TB). The spiRINT 
device may also run WINDOWS® operating systems (e.g., NT®, 2000®, etc.). 
The spiRINT device can input and/or output digital video using a serial digital 
interface according to SMPTE standards (e.g., 259 M, 292M). For example, using 
the SMPTE 292M specification, the spiRINT device can output 10-bit high 
definition video at approxnnately 1.5 Gbps. Use of devices having some or all of 
such features (e.g., features of Accom, Post Impressions, etc.) is described herein 
with respect to a variety of exemplary methods, devices, systems and/or storage 
media. 

Referring again to Fig. 2, once the video data from the telecine has been 
recorded, the recorded video data are converted to another digital format in a 
digital-to-digital conversion block 250. In yet other exemplary methods described 
herein, however, a recorder optionally performs a digital-to-digital conversion. As 
shown in Fig. 2, a computer is configured to perform the digital-to-digital 
conversion. In general, the recorded digital video data are transmitted to the 
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computer using a digital serial interface. Of course, transmission through other 
methods may be used, for example, through a disk-based interface that allows for 
transfer of data from a recorder's disk to a computer. In yet another exemplary 
method, the recording block 240 of the exemplary method 200 is bypassed and an 
analog-to-digital conversion block inputs "unrecorded" digital data from the 
telecine (or the recorder) to the computer for fiirther digital-to-digital conversion. 
In this altemative, for example, a telecine may transmit digital data to a computer 
using a digital serial interface tiiat optionally complies with the SMPTE 292M 
standard or other standard. Of course, in various exemplary methods, audio data 
may also accompany the video data. 

According to the exemplary method 200, a digital-to-digital conversion 
optionally involves converting some or all of the digital video data to a group or a 
series of individual digital image files on a frame-by-frame and/or other suitable 
basis. Of course, in an altemative, not every frame is converted. According to an 
exemplary digital-to-digital conversion, the conversion process converts a frame 
of digital video data to a digital image file and/or frames of digital video data to a 
digital video file. Suitable digital image file formats include, but are not limited 
to, the tag image file format (TIFF), which is a common format for exchanging 
raster graphics (bitmap) images between application programs. The TIFF format 
is capable of describing bilevel, grayscale, palette-color, and full-color image data 
in several color spaces. The TIFF specification includes a number of compression 
schemes such as LZW compression. Joint Photographic Experts Group (JPEG) 
compression, and compression schemes specified by the Intemational Telegraph 
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and Telephone Consultative Committee (CCITT) (e.g., Group 3 and Group 4 
schemes). 

Regarding compression, algorithmic processes for compression generally 
fall into two categories: lossy and lossless. For example, algorithms based on the 
discrete cosine transform (DCT) are lossy whereas lossless algorithms are not 
DCT-based. A baseline JPEG lossy process, which is typical of many DCT-based 
processes, involves encoding by: (i) dividing each component of an input image 
into 8x8 blocks; (ii) performing a two-dimensional DCT on each block; (iii) 
quantizing each DCT coefficient uniformly; (iv) subtracting the quantized DC 
coefficient from the corresponding term in the previous block; and (v) entropy 
coding the quantized coefficients using variable length codes (VLCs). Decoding 
is performed by inverting each of the encoder operations in the reverse order. For 
example, decoding involves: (i) entropy decoding; (ii) performing a 1-D DC 
prediction; (iii) performing an inverse quantization; (iv) performing an inverse 
DCT transform on 8 x 8 blocks; and (v) reconstructing the image based on the 8 x 
8 blocks. While the process is not limited to 8 x 8 blocks, square blocks of 
dimension T x where "n" is an integer, are preferred. A particular JPEG 
lossless coding process uses a spatial-prediction algorithm based on a two- 
dimensional differential pulse code modulation (DPCM) technique. The TIFF 
format supports a lossless Huffman coding process. 

The TIFF specification also includes YCrCb, CMYK, RGB, CIE L*a*b* 
color space specifications. Data for a single image may be striped or tiled, A 
combination of strip-orientated and tile-orientated image data, while potentially 
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possible, is not recommended by the TIFF specification. In general, a high 
resolution image can be accessed more efficiently - and compression tends to 
work better - if the image is broken into roughly square tiles instead of 
horizontally-wide but vertically-narrow strips. Data for multiple images may also 
be tiled and/or striped in a TIFF format; thus, a single TIFF format file may 
contain data for a plurality of images. 

Referring again to Fig. 2, the computer used in the digital-to-digital 
conversion block 250 optionally comprises a computer having video processing 
software. The computer of conversion block 250 can be any suitable computer 
(computing device). Exemplary non-limiting computers include a SILICON 
GRAPHICS® 02+™ computer (Silicon Graphics, Inc., Mountain View, 
Califomia), a SILICON GRAPHICS® 02® computer, a SILICON GRAPHICS® 
ONYX® computer, a SILICON GRAPHICS® 3000® computer, a SILICON 
GRAPHICS® Octane2™ computer or an equivalent thereof. The computer of 
block 250 optionally includes a graphics system. Suitable exemplary, non-limiting 
graphics systems include llie InfiniteReality™ (e.g., IR2, IRS) graphics systems 
(Silicon Graphics, Inc.) and equivalents thereof. An exemplary graphic system 
optionally has multiple processor capability, e.g., consider the IR2 and IRS 
graphics systems. 

The computer of block 250 optionally comprises software such as, but not 
limited to, INFERNO® software (Discreet, Montreal, Quebec, Canada), and 
equivalents thereof. INFERNO® software is suitable for use with film, digital 
cinema, HDTV/DTV, high-resolution video tasks. In combination with a IRS 



27 



0128021335 mi-894USJ*ATAPP 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
U 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 



graphics system, a SILICON GRAPHICS® computer, and/or a SILICON 
GRAPHICS® video input/output (e.g., DMediaPro™ video input/output), 
INFERNO® software offers an enviromnent for high-resolution (e.g., HDTV 
resolution) and feature film visual effects work including real-time 2K film 
playback and 12-bit support and input and/or output of both standard (e.g., 
SMPTE 259M standard) and high-definition (e.g., SMPTE 292M standard) video 
data. Similarly, FLAME® software on a SILICON GRAPHICS® computer (e.g., 
0CTANE®2), including serial digital I/O support for high-definition video, offers 
realtime HDTV I/O for most all popular HDTV formats including 720p, 10801 and 
1080/24p. The SILICON GRAPHICS® DMediaPro™ video input/output devices 
support 4:2:2 and 4:4:4 YCrCb video sampling with 8 or 10 bits per component; 
4:4:4 RGB video sampling with 8 or 10 bits per component; and full sample rate 
for alpha channel at 8 or 10 bits. 

Other systems suitable for use in the digital-to-digital conversion block 250 
include, but are not limited to, systems previously mentioned that are 
manufactured and/or sold by Accom, Inc. and/or Post Impressions, Inc. The 
spiRINT device of Post Impressions uses a "real-time" operating system (OS) 
embedded below a WINDOWS® NT® OS and has a high bandwidth low voltage 
differential signaling (LVDS) bus having dynamically switched bus architecture. 
The "real-time" OS includes a multi-format multi-resolution file system that 
enables files of any resolution and format to co-exist on the media storage and yet 
appear transparent to the NT® OS file system (NTFS). The WSD®/HD device of 
Accom has an OS independent control interface that allows for device control 
fi*om essentially any workstation via, for example, a network connection. 
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Alternatively, the control interface is accessed and run directly on the device, for 
example, with the aid of a monitor (e.g., a display panel, etc.). The Accom and 
Post Impressions devices can input 1.5 Gbps of HD format video data using a 
SMPTE 292M standard serial digital interface or 0.27 Gbps of SD format video 
data using a SMPTE 259M standard serial digital interface. Thus, such devices 
may interface a telecine and/or a recorder and/or, as mentioned previously, operate 
as a recorder. Use of such devices is further described in accordance with various 
exemplary methods, devices, systems and/or storage media that follow. 

As already mentioned, in the digital-to-digital conversion block 250, 
software and a computer convert digital video data to a digital image file(s) or 
digital video file(s). Sometimes, such a process is referred to as "capture", 
wherein images are captured from digital video data - in either instance, a digital- 
to-digital conversion occurs. According to the exemplary method 200, digital 
video data from the telecine and/or the recorder may be compressed and/or 
uncompressed. The digital-to-digital conversion is optionally performed on a 
frame-by-frame basis, wherein each frame of digital video data transmitted from a 
telecine or a recorder is converted to a digital image file. Furthermore, a one-to- 
one correspondence is optionally maintained between each original analog (or 
digital) frame and a digital unage file. However, a 3:2 pulldown or other type of 
pulldown or editing is also possible. A digital video file may also maintain a one- 
to-one correspondence between each original frame and frames in the digital video 
file; of course, other options also exist, such as, but not limited to, a 3:2 pulldown. 
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In an exemplary, non-limiting digital-to-digital conversion process (see, 
e.g., conversion block 250), digital video data are converted to image files, which 
are optionally recorded on a recording medium. For example, digital video data 
are transmitted according to the SMPTE 292M specification to a computer 
wherein the video data are converted to TIFF format files on a fi-ame-by-frame or 
other suitable basis, wherein, during and/or after the conversion, the TIFF format 
files are recorded on digital linear tape (DLT). DLT is a form of magnetic tape and 
drive system used for storage of data. A compression algorithm, known as Digital 
Lempel Ziv 1 (DLZl), facilitates storage and retrieval of data at high speeds and 
in large quantities. A DLT drive records data on a tape in dozens of straight-line 
(linear) tracks, usually 128 or 208. Some tape cartridges can hold 70 gigabytes 
(GB) of data when compression is used. A variant of DLT technology, called 
SuperDLT, makes it possible to store upwards of 100 GB on a single tape 
cartridge. A SuperDLT drive can transfer data at speeds of up to 10 megabytes per 
second (MBps). Exemplary alternative recording systems include linear tape open 
(LTO) drives, advanced intelligent tape (AIT) drives, and Mammoth drives. 

Referring again to Fig. 2, a second conversion digital-to-digital conversion 
block 260 is shown. In this conversion block 260, digital data, e.g., produced by 
the conversion block 250, are converted to a format suitable for at least one file 
and/or at least one data stream suitable for processing by a computer to thereby 
produce a video display. For example, in an exemplary, non-limiting conversion 
block (see, e.g., conversion block 260), a computer receives digital image files 
from a tape drive or another computer in a TIFF format. The TIFF format files are 
then converted to an audio video interleaved (AVI) format file, which is suitable 
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for a file and/or a stream and/or fiuther conversion to another format as a file 
and/or a stream. For example, an exemplary, non-limiting conversion block 
converts a AVI format file to a WINDOWS MEDIA™ format file and/or at least 
one data stream. 

The AVI file format is a file format for digital video and audio for use with 
WINDOWS® OSs and/or other OSs. According to the AVI format, blocks of 
video and audio data are interspersed together. Although an AVI format file can 
have "n" number of streams, the most common case is one video stream and one 
audio stream. The stream format headers define the format (including 
compression) of each stream. 

AVI format files may be made in several different ways. For example, 
VIDEDIT™ software or WINDOWS® MOVIE MAKER^^ software (Microsoft 
Corporation) can create an AVI format file from image files. The VIDEDIT™ 
software uses bitmap image files, thus, TIFF format files need to be converted first 
to bitmap files. Once converted, VIDEDIT™ software assembles the bitmap 
images into an AVI format file, typically in an animation sequence. VIDEDIT™ 
can delete frames or add other frames or sequences. AVI format files can also be 
cropped or resized before being saved full sized or compressed. Such facilities are 
also provided by WINDOWS® MOVIE MAKER™ software, which can also use 
TIFF format files to create an AVI format file. 

Referring again to Fig. 2, a primary function of the conversion block 260 is 
to produce a file and/or at least one data stream. Such a file and/or stream may be 
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in a WINDOWS MEDIA format, which is a format capable of use in, for 
example, streaming audio, video and text from a server to a client computer. A 
WINDOWS MEDIA™ format file may also be stored and played locally. In 
general, a format may include more than just a file format and/or stream format 
specification. For example, a format may include codecs. Consider, as an 
example, the WINDOWS MEDIA™ format, which comprises audio and video 
codecs, an optional integrated digital rights management (DRM) system, a file 
container, etc. As referred to herein, a WINDOWS MEDIA™ format file and/or 
WINDOWS MEDIA™ format stream have characteristics of files suitable for use 
as a WINDOWS MEDIA™ format container file. Details of such characteristics 
are described below. In general, the term "format" as used for files and/or streams 
refers to characteristics of a file and/or a stream and not necessarily characteristics 
of codecs, DRM, etc. Note, however, that a format for a file and/or a stream may 
include specifications for inclusion of information related to codec, DRM, etc. 

A block diagram of an exemplary conversion process for converting 
information to a suitable file and/or stream format 300 is shown in Fig. 3. 
Referring to Fig. 3, in the exemplary conversion process 300, a conversion block 
312 accepts information from a metadata block 304, an audio block 306, a video 
block 308, and/or a script block 310. The information is optionally contained in 
an AVI format file and/or in a stream; however, the information may also be in an 
uncompressed WINDOWS MEDIA™ format or other suitable format. In an 
audio processing block 314 and in a video processing block 318, the conversion 
block 312 performs audio and/or video processing. Next, in an audio codec block 
322 and in a video codec block 326, the conversion block 312 compresses the 
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processed audio, video and/or other information and outputs the compressed 
information to a file container 340. Before, during and/or after processing and/or 
compression, a rights management block 330 optionally imparts information to the 
file container block 340 wherein the information is germane to any associated 
rights, e.g., copyrights, trademark rights, patent, etc., of the process or the 
accepted information. 

The file container block 340 typically stores file information in a single file. 
Of course, information may be streamed in a suitable format rather than 
specifically "stored". An exemplary, non-limiting file and/or stream has a 
WINDOWS MEDIA™ format. The term "WINDOWS MEDIA™ format", as 
used throughout, includes the active stream format and/or the advanced systems 
format, which are typically specified for use as a file container format. The active 
stream format and/or advanced systems format may include audio, video, 
metadata, index commands and/or script commands (e.g., URLs, closed 
captioning, etc.). In general, information stored in a WINDOWS MEDIA™ file 
container, will be stored in a file having a file extension such as .wma, .wmv, or 
.asf; streamed information may optionally use a same or a similar extension(s). 

In general, a file (e.g., according to a file container specification) contains 
data for one or more streams that can form a multimedia presentation. Stream 
delivery is typically synchronized to a common timeline. A file and/or stream may 
also include a script, e.g., a caption, a URL, aad/or a custom script command. As 
shown in Fig. 3, the conversion process 300 uses at least one codec or 
compression algorithm to produce a file and/or at least one data stream. In 
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particular, such a process may use a video codec or compression algorithm and/or 
an audio codec or compression algorithm. Furthermore, the conversion block 260 
optionally supports compression and/or decompression processes that can utilize a 
plurality of processors, for example, to enhance compression, decompression, 
and/or execution speed of a file and/or a data stream. 

One suitable video compression and/or decompression algorithm (or codec) 
is entitled MPEG-4 v3, which was originally designed for distribution of video 
over low bandwidth networks using high compression ratios (e.g., see also MPEG- 
4 v2 defined in ISO MPEG-4 document N3056). The MPEG-4 v3 decoder uses 
post processors to remove "blockiness", which improves overall video quality, and 
supports a wide range of bit rates firom as low as 10 kbps (e.g., for modem users) 
to 10 Mbps or more. Another suitable video codec uses block-based motion 
predictive coding to reduce temporal redundancy and transform coding to reduce 
spatial redundancy. 

A suitable conversion software package that uses codecs is entitled 
WINDOWS MEDIA™ Encoder. The WINDOWS MEDIA™ Encoder software 
can compress live or stored audio and/or video content into WINDOWS 
MEDIA™ format files and/or data streams (e.g., such as the process 300 shown in 
Fig. 3). This software package is also available in the form of a software 
development kit (SDK). The WINDOWS MEDIA™ Encoder SDK is one of the 
main components of the WINDOWS MEDIA™ SDK. Other components include 
the WINDOWS MEDIA™ Services SDK, the WINDOWS MEDIA™ Format 
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SDK, the WINDOWS MEDIA'^ Rights Manager SDK, and the WINDOWS 
MEDIA™ Player SDK. 



The WINDOWS MEDIA'^ Encoder 7.1 software optionally uses an audio 
codec entitled WINDOWS MEDIA™ Audio 8 (e.g., for use in the audio codec 
block 322) and a video codec entitled WINDOWS MEDIA™ Video 8 codec (e.g., 
for use in the video codec block 326). The Video 8 codec uses block-based motion 
predictive coding to reduce temporal redundancy and transform coding to reduce 
spatial redundancy. Of course, later codecs, e.g.. Video 9 and Audio 9, are also 
suitable. These aforementioned codecs are suitable for use in real-time capture 
and/or streaming applications as well as non-real-time applications, depending on 
demands. In a typical application, WINDOWS MEDIA™ Encoder 7.1 software 
uses these codecs to compress data for storage and/or streaming, while 
WINDOWS MEDIA™ Player software decompresses the data for playback. 
Often, a file or a stream compressed with a particular codec or codecs may be 
decompressed or played back using any of a variety of player software. In general, 
the player software requires knowledge of a file or a stream compression codec. 

The Audio 8 codec is capable of producing a WINDOWS MEDIA™ 
format audio file of the same quality as a MPEG-1 audio layer-3 (MP3) format 
audio file, but at less than approximately one-half the size. While the quality of 
encoded video depends on the content being encoded, for a resolution of 640 pixel 
by 480 line, a frame rate of 24 fps and 24 bit depth color, the Video 8 codec is 
capable of producing 1:1 (real-time) encoded content in a WINDOWS MEDIA™ 
format using a computer having a processor speed of approximately 1 GHz. The 
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same approximately 1 GHz computer would encode video having a resolution of 
1280 pixel by 720 line, a frame rate of 24 fps and 24 bit depth color in a ratio of 
approximately 6:1 and a resolution of 1920 pixel by 1080 line, a frame rate of 24 
fps and 24 bit depth color in a ratio of approximately 12:1 (see also the graph of 
Fig. 14 and the accompanying description). Essentially, the encoding process in 
these examples is processor speed limited. Thus, an approximately 6 GHz 
processor computer can encode video having a resolution of 1280 pixel by 720 
line, a frame rate of 24 j^s and 24 bit depth color in real-time; likewise, an 
approximately 12 GHz computer can encode video having a resolution of 1920 
pixel by 1080 line, a frame rate of 24 fps and 24 bit depth color in real-time. 
Overall, the YidQO 8 codec and ftmctional equivalents thereof are suitable for use 
in converting, streaming and/or downloading digital data. Of course, according to 
various exemplary methods, devices, systems and/or storage media described 
herein, video codecs other than the Video 8 may be used. 

The WINDOWS MEDIA™ Encoder 7.1 supports single-bit-rate (or 
constant) streams and/or variable-bit-rate (or multiple-bit-rate) streams. Single- 
bit-rates and variable-bit-rates are suitable for some real-time capture and/or 
streaming of audio and video content and support of a variety of connection types, 
for example, but not limited to, 56 Kbps over a dial-up modem and 500 Kbps over 
a cable modem or DSL line. Of course, other higher bandwidth connections types 
are also supported and/or supportable. Thus, support exists for video profiles 
(generally assuming a 24 bit color depth) such as, but not limited to, DSL/cable 
delivery at 250 Kbps, 320 x 240, 30 Q)s and 500 Kbps, 320 x 240, 30 fps; LAN 
delivery at 100 Kbps, 240 x 180, 15 fps; and modem delivery at 56 Kbps, 160 x 
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I2O5 15 ^s. The exemplary Video 8 and Audio 8 codecs are suitable for 
supporting such profiles wherein the compression ratio for video is generally at 
least approximately 50:1 and more generally in the range of approximately 200:1 
to approximately 500:1 (of course, higher ratios are also possible). For example, 
video having a resolution of 320 pixel by 240 line, a firame rate of 30 fps and a 
color depth of 24 bits requkes approximately 55 Mbps; thus, for DSL/cable 
delivery at 250 Kbps, a compression ratio of at least approximately 220:1 is 
required. Consider another example, a 1280 x 720, 24 :^s profile at a color bit 
depth of 24 corresponds to a rate of approximately 0.53 Gbps, Compression of 
approximately 500:1 reduces this rate to approximately 1 Mbps. Of course, 
compression may be adjusted to target a specific rate or range of rates, e.g., 0.1 
Mbps, 0.5 Mbps, 1.5 Mbps, 3 Mbps, 4.5 Mbps, 6 Mbps, 10 Mbps, 20 Mbps, etc. 
In addition, where bandwidth allows, compression ratios less than approximately 
200:1 may be used, for example, compression ratios of approximately 30:1 or 
approximately 50:1 may be suitable. Of course, while an approximately 2 Mbps 
data rate is available over many LANs, even a higher speed LAN may require 
fiirther compression to facilitate distribution to a plurality of users (e.g., at 
approximately the same time). Again, while these examples refer to the Video 8 
and/or Audio 8 codecs, use of other codecs is also possible. 

The Video 8 and Audio 8 codecs, when used with the WINDOWS 
MEDIA™ Encoder 7.1 may be used for capture, compression and/or streaming of 
audio and video content in a WINDOWS MEDIA™ format. Conversion of an 
existing video file(s) (e.g., AVI format files) to the WINDOWS MEDIA™ file 
format is possible with WINDOWS MEDIA™ 8 Encoding Utility software. The 
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WINDOWS MEDIA™ 8 Encoding Utility software supports "two-pass" and 
variable-bit-rate encoding. The WINDOWS MEDIA™ 8 Encoding Utility 
software is suitable for producing content in a WINDOWS MEDIA^^ format that 
can be downloaded and played locally. 

As already mentioned, the WINDOWS MEDIA™ format optionally 
includes the active stream format and/or the advanced systems format. Various 
features of the active stream format are described in U.S. Patent No. 6,041,345, 
entitled "Active stream format for holding multiple media streams", issued March 
21, 2000, and assigned to Microsoft Corporation ('345 patent). The '345 patent is 
incorporated herein by reference for all purposes, particularly those related to file 
formats and/or stream formats. The '345 patent defines an active stream format 
for a logical structure that optionally encapsulates multiple data streams, wherein 
the data streams may be of different media (e.g., audio, video, etc.). The data of 
the data streams is generally partitioned into packets that are suitable for 
transmission over a transport medium (e.g., a network, etc.). The packets may 
include error correcting information. The packets may also mclude clock Ucenses 
for dictating the advancement of a clock when the data streams are rendered. The 
active stream format can facilitate flexibility and choice of packet size and bit rate 
at which data may be rendered. Error concealment strategies may be employed in 
the packetization of data to distribute portions of samples to multiple packets. 
Property information may also be repUcated and stored in separate packets to 
enhance error tolerance. 
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In general, the advanced systems format is a file format used by 
WINDOWS MEDIA™ technologies and it is generally an extensible format 
suitable for use in authoring, editing, archiving, distributing, streaming, playing, 
referencing and/or otherwise manipulating content (e.g., audio, video, etc). Thus, 
it is suitable for data delivery over a wide variety of networks and is also suitable 
for local playback. In addition, it is suitable for use with a transportable storage 
medium, as described in more detail below. As mentioned, a file container (e.g., 
the file container 340) optionally uses an advanced systems format, for example, 
to store any of the following: audio, video, metadata (such as the file's title and 
author), and index and script commands (such as URLs and closed captioning); 
which are optionally stored in a single file. Various features of the advanced 
systems format appear in a document entitled "Advanced Systems Format (ASF)" 
fi-om Microsoft Corporation (Doc. Rev. 01.13.00e - current as of 01.23.02). This 
document is a specification for the advanced systems format and is available 
through the Microsoft Corporation Web site ( www.microsoft.comy The 
"Advanced Systems Format (ASF)" document (sometimes referred to herein as the 
"ASF specification") is incorporated herein by reference for all purposes and, in 
particular, purposes relating to encoding, decoding, file formats and/or stream 
formats. 

An ASF file typically includes three top-level objects: a header object, a 
data object, and an index object. The header object is commonly placed at the 
beginning of an ASF file; the data object typically follows the header object; and 
the index object is optional, but it is usefiil in providing time-based random access 
into ASF files. The header object generally provides a byte sequence at the 
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beginning of an ASF file (e.g., a GUID to identify objects and/or entities within an 
ASF file) and contains information to interpret information within the data object. 
The header object optionally contains metadata, such as, but not limited to, 
bibliographic information, etc. 

An ASF file and/or stream may include information such as, but not limited 
to, the following: format data size (e.g., number of bytes stored in a format data 
field); image width (e.g., width of an encoded image in pixels); rniage height (e.g., 
height of an encoded image in pixels); bits per pixel; compression ID (e.g., type of 
compression); image size (e.g., size of an image in bytes); horizontal pixels per 
meter (e.g., horizontal resolution of a target device for a bitmap in pixels per 
meter); vertical pixels per meter (e.g., vertical resolution of a target device for a 
bitmap in pixels per meter); colors used (e.g., number of color indexes in a color 
table that are actually used by a bitmap); important colors (e.g., number of color 
indexes for displaying a bitmap); codec specific data (e.g., an array of codec 
specific data bytes). 

The ASF also allows for inclusion of commonly used media types, which 
may adhere to other specifications. In addition, a partially downloaded ASF file 
may still fimction (e.g., be playable), as long as required header information and 
some complete set of data are available. 

As mentioned, tiie WINDOWS MEDIA™ 8 Encodmg Utility is capable of 
encoding content at variable bit rates. In general, encoding at variable bit rates 
may help preserve image quality of the original video because the bit rate used to 
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encode each frame can fluctuate, for example, with the complexity of the scene 
composition. Types of variable bit rate encoding include quality-based variable bit 
rate encoding and bit-rate-based variable bit rate encoding. Quality-based variable 
bit rate encoding is typically used for a set desired image quality level. In this 
type of encoding, content passes through the encoder once, and compression is 
applied as the content is encountered. This type of encoding generally assures a 
high encoded image quality. Bit-rate-based variable bit rate encoding is useful for 
a set desired bit rate. In this type of encoding, the encoder reads through tiie 
content first in order to analyze its complexity and then encodes the content in a 
second pass based on the first pass information. This type of encoding allows for 
control of output file size. As a further note, generally, a source file must be 
uncompressed; however, compressed (e.g., AVI format) files are supported if an 
image compression manager (ICM) decompressor software is used. 

Use of the Video 8 codec (or essentially any codec) due to compression 
and/or decompression computations places performance demands on a computer, 
in particular, on a computer's processor or processors. Demand variables include, 
but are not limited to, resolution, frame rate and bit depth. For example, a media 
player relying on the Video 8 codec and executing on a computer with a processor 
speed of approximately 0.5 GHz can decode and play encoded video (and/or 
audio) having a video resolution of 640 pixel by 480 line, a frame rate of 
approximately 24 fps and a bit depth of approximately 24. A computer with a 
processor of approximately 1.5 GHz could decode and play encoded video (and/or 
audio) having a video resolution of 1280 pixel by 720 line, a frame rate of 
approximately 24 fps and a bit depth of approximately 24; while, a computer with 
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a processor of approximately 3 GHz could decode and play encoded video (and/or 
audio) having a video resolution of 1920 pixel by 1080 line, a frame rate of 
approximately 24 fps and a bit depth of approximately 24 (see also the graph of 
Fig. 14 and the accompanying description). 

A block diagram of an exemplary compression and decompression process 
400 is shown in Fig. 4. In this exemplary compression and decompression process 
400, an 8 pixel x 8 pixel image block 404 from, for example, a frame of a 1920 
pixel X 1080 line image, is compressed in a compression block 408, to produce a 
bit stream 412. The bit stream 412 is then (locally and/or remotely, e.g., after 
streaming to a remote site) decompressed in a decompression block 416. Once 
decompressed, the 8 pixel x 8 pixel image block 404 is ready for display, for 
example, as a pixel by line image. 

Note that the compression block 408 and the decompression block 416 
include several intemal blocks as well as a shared quantization table block 430 and 
a shared code table block 432 (e.g., optionally containing a Huffman code table or 
tables). These blocks are representative of compression and/or decompression 
process that use a DCT algorithm (as mentioned above) and/or other algorithms. 
For example, as shown in Fig. 4, a compression process that uses a transform 
algorithm generally involves performing a transform on a pixel image block in a 
traQsform block 420, quantizing at least one transform coefficient in a quantization 
block 422, and encoding quantized coefficients in a encoding block 424; whereas, 
a decompression process generally involves decoding quantized coefficients in a 
decoding block 444, dequantizing coefficients in a dequantization block 442, and 
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performing an inverse transform in an inverse transform block 440. As mentioned, 
the compression block 408 and/or the decompression block 416 optionally include 
other functional blocks. For example, the compression block 408 and the 
decompression block 416 optionally include functional blocks related to image 
block-based motion predictive coding to reduce temporal redundancy and/or other 
blocks to reduce spatial redundancy. In addition, blocks may relate to data 
packets. Again, the WINDOWS MEDIA™ format is typically a packetized 
format in that a bit stream, e.g., the bit stream 412, would contain information in a 
packetized form. In addition, header and/or other information are optionally 
included wherein the information relates to such packets, e.g., padding of packets, 
bit rate and/or other format information (e.g., error correction, etc.). In general, 
the exemplary method for producing a stream 200 produces a bit stream such as 
the bit stream 412 shown in Fig. 4. 

Compression and/or decompression processes may also include other 
features to manage the data. For example, sometimes every frame of data is not 
fully compressed or encoded. According to such a process frames are typically 
classified, for example, as a key frame or a delta frame. A key frame may 
represent frame that is entirely encoded, e.g., similar to an encoded still image. 
Key frames generally occur at intervals, wherein each frame between key frames 
is recorded as the difference, or delta, between it and previous frames. The 
number of delta frames between key frames is usually determinable at encode time 
and can be manipulated to accommodate a variety of circumstances. Delta frames 
are compressed by their very nature. A delta frame contains information about 
image blocks that have changed as well motion vectors (e.g., bidirectional, etc.), or 



Ue& Hayes, PLUZ 



43 



012802 J 33SAdSI-894US,PATjiPP 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 



infomiation about image blocks that have moved since the previous frame. Using 
these measurements of change, it might be more efficient to note the change in 
position and composition for an existing image block than to encode an entirely 
new one at the new location. Thus delta frames are most compressed in situations 
where the video is very static. As akeady explained, compression typically 
involves breaking an image into pieces and mathematically encoding the 
information in each piece. In addition, some compression processes optimize 
encoding and/or encoded information. Further, other compression algorithms use 
integer transforms that are optionally approximations of the DCT, such algorithms 
may also be suitable for use in various exemplary methods, devices, systems 
and/or storage media described herein. In addition, a decompression process may 
also include post-processing. 

Referring again to Fig. 2, the conversion process 260 optionally produces a 
bit stream capable of carrying variable-bit-rate and/or constant-bit-rate video 
and/or audio data in a particular format. As already discussed, bit streams are 
often measured in terms of bandwidth and in a transmission unit of kilobits per 
second (Kbps), millions of bits per second (Mbps) or billions of bits per second 
(Gbps). For example, an integrated services digital network line (ISDN) type T-1 
can, at the moment, deliver up to 1.544 Mbps and a type El can, at the moment, 
deliver up to 2.048 Mbps. Broadband ISDN (BISDN) can support transmission 
from 2 Mbps up to much higher, but as yet unspecified, rates. Another example is 
known as digital subscriber line (DSL) which can, at the moment, deliver up to 8 
Mbps. A variety of other examples exist, some of which can transmit at bit rates 
substantially higher than those mentioned herein. For example, Intemet2 can 



Lee &Hayes,?UX: 



44 



0128021335 MSI-894US.PATAPP 



5» 

1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 



support data rates in the range of approximately 100 Mbps to several gigabytes per 
second. The exemplary method 200 optionally provides bit streams at a variety of 
rates, including, but not limited to, approximately 1,5 Mbps, 3 Mbps, 4.5 Mbps, 6 
Mbps, and 10 Mbps. Such bit streams optionally include video data having a pixel 
by line format and/or a frame rate that corresponds to a common digital video 
format as listed in Table 1. 

Fig. 5 shows a block diagram of an exemplary method 500 for producing a 
bit stream. In this exemplary method 500, the resulting bit stream has upon 
decompression and display, for example, a 1280 pixel x 720 line format, a frame 
rate of 24 fps and a bit depth of approximately 24. According to the method 500, 
in a process block 504, a photographic film is processed. Of course, the film may 
also have an audio track recorded on the film and/or on another medium. Next, in 
a conversion block 508, a telecine, or other structurally and/or functionally 
equivalent thereof, converts the processed photographic fihn to a digital data 
stream, which optionally includes audio data. For example, the digital data stream 
may have a SMPTE 292M digital video data stream having a 1920 pixel by 1080 
line format, a frame rate of approximately 24 fps and a bit depth of approximately 
24. Following the conversion block 508, in a record block 512, a recorder records 
the digital data stream to a suitable recording medium. For example, the record 
block 512 may optionally use a recorder capable of recording to a D5 and/or a D6 
cassette tape or to a disk or disk array. After the record block 512, the recorded 
digital video data are converted to a particular format suitable for streaming and/or 
storing in a conversion block 516. For example, a SILICON GRAPHICS® 
computer and INFERNO® software is optionally used to accept a digital data 
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stream having a SMPTE 292M specification format, such as from a magnetic 
cassette tape played using a recorder. Alternatively, the exemplary method 500 
may use a SILICON GRAPHICS® computer to accept a stream directly from a 
telecine and thereby bypass the record block 512. In yet another altemative, a 
DDR device records the digital data stream to a disk array. In this altemative, the 
data are optionally recorded to a disk array in a QUICKTIME® format and/or a 
WINDOWS MEDIA™ format. 

Wherein the conversion block 516 optionally converts the digital video data 
stream to image files, such as, but not limited to, TIFF files, the image files may 
retain the original pixel by line format and/or frame rate. Alternatively, the image 
data may be scaled and/or frames omitted; of course, these and/or other operations 
may be performed in the conversion block 516. Following the conversion block 
516, in a streaming block 520, the converted video data are streamed for 
decompression, display and/or storage. 

The exemplary method 500 may also optionally convert 1920 pixel by 1080 
line digital data to a format suitable for storage and then scale the stored data to a 
1280 pixel by 720 line fomiat. In this example, after scaling, the stored data is 
optionally converted to WINDOWS MEDIA™ format and streamed in at least one 
bit stream having, for example, but not limited to, a bandwidth of approximately 
1.5 Mbps, 3 Mbps, 6 Mbps, and/or 10 Mbps. Note that according to aspects of 
other exemplary methods described herein, conversion to a particular format does 
not necessarily involve compression, for example, consider conversion from an 
uncompressed QUICKTIME® format to an uncompressed WINDOWS MEDIA™ 
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format. Such a conversion is optionally based on a conversion of header 
information. 

In the exemplary method 500, scaling is optionally performed to account 
for processing power of downstream destinations, e.g., clients. For example, a 900 
MHz PENTIUM® III processor (Intel Corporation, Delaware) m a computer with 
appropriate buss architecture and a VGA output card can produce consistent play 
of a 0.75 Mbps stoeam having an 853 pixel by 486 pixel image format, a frame 
rate of 24 fps, and a bit depth of approximately 24 (e.g., "true color"). Dual LI 
Ghz PENTIUM® III processors in a computer or a single 1.4 GHz AMD® 
processor (Advanced Micro Devices, Incorporated, Delaware) in a computer can 
consistently be used to decode and play of a stream having a 1280 pixel by 720 
line format, a frame rate of 24 fps and a bit depth of approximately 24 (e.g., 'true 
color") while dual I A GHz AMD® processors in a computer can be used to 
decode and play a stream having a 1920 pixel by 1080 line image format, a frame 
rate of 24 ^s and a bit depth of approximately 24 (e.g., "true color"). Of course, 
other arrangements are possible, including single processor computers having 
processor speeds in excess of 1 GHz (also see the graph of Fig. 14 and the 
accompanying description). 

Referring to Fig. 6, a digital storage and/or sliucturing device 610 is shown. 
While Fig. 6 shows functional blocks in a device, various functional blocks 
optionally appear as a system wherein more than one computer (e.g., computing 
device) is used. The digital storage and/or structuring device 610 optionally 
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includes some or all features of the aforementioned devices of Accom, Inc. and/or 
Post Impressions, Inc. The device 610 may also include some or all features of 
other hardware and/or software described herein. Thus, the digital storage and/or 
structuring device 610 is optionally capable of recording video data from a 
telecine and/or other analog-to-digital conversion device (e.g., a digital camera). 
The digital storage and/or structuring device is optionally also capable of receiving 
digital video data from other sources (e.g., a recorder/player). As shown in Fig. 6, 
this device 610 includes a variety of frinctional hardware and/or software blocks, 
some of which may be optional. The blocks include a digital serial interface (DSI) 
block 614 for receiving and/or sending video data via a digital serial interface. 
The DSI block 614 may receive and/or send digital video data transmitted 
according to an SMPTE standard and/or other standards. A processor block 618 
performs various computational tasks typically related to other fimctional blocks. 
A RAM block 622 optionally stores video data prior to storage in a storage block 
624. A structure block 626 optionally structures video data from the RAM block 
610 or from another block prior to storage in the storage block 624. For example, 
the device 610 may receive video data via the DSI block 614, transmit the data to 
the RAM block 622 for storage in RAM and then structure the video data in the 
structure block 626 to allow for more efficient storage of the video data in the 
storage block 626. Accordingly, the structure block 626 may structure the data 
according to a format, typically suitable for storage. Such formats include, but are 
not limited to, a WINDOWS MEDIA™ format. In this particular example, the 
data is optionally in an "uncompressed" form, in that, it has not been compression 
encoded. In one particular example, the structure block 626 structures video data 
in a particular format and stores the structured data to a disk or disks. Structuring 
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may also include structuring of format information (e.g., contained in a file 
header) to other information associated with another format. Such structuring may 
effectively produce a WINDOWS MEDIA™ format file and/or stream suitable for 
encoding by a WINDOWS MEDIA™ encoder (e.g., compression encoding). 
Further, structuring may also include encoding, e.g., to thereby produce a file 
and/or a stream suitable for decompression or decoding. 

A scaler block 630 optionally scales video data prior to and/or after storage 
of video data. The scaler block 630 optionally scales video resolution (e.g., pixel 
and/or line) and/or fi-ame rate (e.g., drops frames). In addition, the scaler block 
630 may also scale and/or alter color information, potentially according to a color 
space specification and/or sampling format (e.g., reducing bit depth). The scaler 
block 630 optionally comprises scaling software. Such software is optionally 
ADOBE® PREMIER® software (Adobe Systems, Inc., San Jose, CaUfomia). 
The ADOBE® PREMIER® software can edit digital video data ui a variety 
formats, including QUICKTIME® format, WINDOWS MEDIA™ format, and 
AVI format In an exemplary system, a scaler block resides on a separate 
computer that optionally accepts video data from a device such as the device 610 
shown m Fig. 6. Such a system may also be capable of ttansmitting scaled video 
data, whether encoded or unencoded, in a variety of formats. 

The device 610 optionally includes an encode block 634 that can encode 
video data. For example, the encode block 634 can encode video data stored m 
the storage block 624. The encode block 634 optionally includes software 
components for encoding. For example, the encode block 634 optionally includes 
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WINDOWS MEDIA^^ technology components that operate on a WINDOWS® 
OS or other OS. According to an exemplary system, the encoder block 634 is 
optionally executed on a separate computer in communication with the device 610 
wherein the separate computer optionally includes storage and/or a 
communication interface. The encoded video data is then optionally stored in the 
storage block 624 and/or transmitted via a network block 638. As mentioned, the 
encode block 634 is optionally included in the structure block 626; thus, 
structuring optionally includes encoding. While the description largely pertains to 
video, it is understood that often audio data will accompany the video data and 
that the WINDOWS MEDIA™ format and/or other formats (e.g., QUICKTIME® 
format, etc.) can be used for, and may include, both video and audio data. 

Fig. 7 shows a block diagram illustrating an exemplary method for 
structuring and storing video data 700. In a reception block 704, a device (e.g., the 
device 610 of Fig. 6) receives digital video data via a digital serial interface. Next, 
in a structuring block 708, the device structures the digital video data in a manner 
that facilitates storage of the video data onto a storage medium (e.g., in a storage 
block 712). For example, the device may structure the video data to facilitate 
storage of the data onto a disk or a disk array. As mentioned previously, such 
structuring optionally includes structuring to a WINDOWS MEDIA™ format. 
Once the video data is stored onto a storage medium, then, in a scale block 716, 
the device optionally scales the data in manner that may facilitates distribution 
and/or playback of the video data. In an exemplary system, such scaling 
optionally occurs on another computer in communication with a device such as the 
device 610. 
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Finally, the scaled data is transmitted via a network or other transmission 
means to a downstream client or clients, in a transmit block 724. Or altematively, 
the scaled data is encoded, for example, using a WINDOWS MEDIA™ encoder. 
The device may receive 1920 pixel by 1080 line resolution video at a rate of 
approximately 1.5 Gbps, structure and store this data in or near real-time, scale the 
data to fit a particular downstream client and then transmit the data to the 
downstream client, or altematively, encode the data prior to transmission. The 
device may optionally save scaled data and then transmit the already saved scaled 
data and/or scale data on the fly or as demanded. 

A block diagram of another exemplary method for storing and/or 
structuring data 800 is shown in Fig. 8. In a reception block 804, a device (e.g., 
the device 610 of Fig. 6) receives digital video data via a digital serial interface. 
Next, in a structuring block 808, the device structures the digital video data in a 
manner that facilitates storage of the video data onto a storage medium (e.g., in a 
storage block 812). For example, the device may structure the video data to 
facilitate storage of the data onto a disk or a disk array. As mentioned previously, 
such structuring optionally includes structuring to a WINDOWS MEDIA^^ 
format. Once the video data is stored onto a storage medium, then, in an encode 
block 816, the device optionally encodes the data in manner that may facilitate 
distribution and/or playback of the video data. Finally, the encoded data is 
transmitted via a network to a downstream client or clients. For example, the 
device may receive 1920 pixel by 1080 line resolution video at a rate of 
approximately 1.5 Gbps, structure and store this data in or near real-tune, encode 
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the data to fit a particular downstream client and then transmit the data to the 
downstream client. The device may optionally save encoded data and then 
transmit the already saved encoded data and/or encode data on the fly or as 
demanded. Encoded data is optionally transmitted as a complete file or as a data 
stream. In a particular example, the encoded data is in a WINDOWS MEDIA™ 
format. 

An exemplary method that makes use of features of the device 610 and of 
the exemplary methods 700 and 800 receives digital video data having a resolution 
of approximately 1920 pixel by approximately 1080 lines. Next, the data is 
structured in a format suitable for storage. Once stored, a computer having scaling 
software accesses the stored data and scales the resolution to approximately 1280 
pixel by approximately 720 lines. After scaling, a software block, optionally 
operating on the same computer as the scaling software, structures the data into 
another format and then encodes the data. For example, the computer optionally 
structures the data in a WINDOWS MEDIA™ format and encodes the data using 
a WINDOWS MEDIA™ codec. 

In the exemplary methods, devices and/or systems referred to in Figs. 6-8, 
a device (e.g., the device 610 of Fig. 6) optionally transmits stored video data to a 
CD recorder and/or a DVD recorder. The CD and/or DVD recorder then records 
the data, which is optionally encoded or compressed and/or scaled to facilitate 
playback on a CD and/or DVD player. DVD players can typically play data at a 
rate of 10 Mbps; however, future players can be expected to play data at higher 
rates, e.g., perhaps 500 Mbps. In this particular example, the device scales the 
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video data according to a DVD player specification (e.g., according to a data rate) 
and transmits the scaled data to a DVD recorder. The resulting DVD is then 
playable on a DVD player having the player specification. According to such a 
method, encoding or compression is not necessarily required in that scaling 
achieves a suitable reduction in data rate. In general, scaling is a process that does 
not rely on a process akin to compression/decompression (or encoding/decoding) 
in that information lost during scaling is not generally expected to be revived 
downstream. Where encoding or compression is used, a suitable compression 
ratio is used to fit the content onto a DVD disk or other suitable disk. 

Referring to Fig. 9, a block diagram of an exemplary method 900 for 
converting photographic film to a WINDOWS MEDIA™ format stream and 
delivering the stream is shown. In a conversion block 904, analog photographic 
film is converted to a digital data stream. Of course, the film may also have an 
audio track recorded on the film and/or on another medium. In such instances, the 
audio track is optionally converted to a digital data stream. In a subsequent 
conversion block 908, the digital data stream is converted to a WINDOWS 
MEDIA™ format (WMF) stream, wherein the WINDOWS MEDIA™ format 
stream optionally has a variable-bit-rate and/or a constant-bit-rate. Next, in a 
delivery block 912, the WINDOWS MEDIA™ format stream is delivered in a 
variety of manners. For example, the WINDOWS MEDIA™ format stream is 
optionally delivered as IP data in a digital TV transmission, via a dish network 
(e.g.. Direct PC, etc.), as a CD and/or a DVD, and/or via a high speed dial-up 
connection. While a WINDOWS MEDIA™ format stream is shown, such an 
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exemplary method is not necessarily limited to use of a WINDOWS MEDIA'^ 
format. 

Once an encoded stream and/or file are delivered, a computer having 
appropriate decompression (or decoding) software (e.g., WINDOWS MEDIA™ 
technology software) may play the video and/or audio information encoded in the 
encoded format stream and/or file. For example. Fig. 10 shows a diagram of an 
exemplary method 1000 for playing video and/or audio information delivered in 
an encoded format. According to this exemplary method 1000, a computer 1004 
havmg decompression software receives digital data in an encoded format as a 
stream and/or as file. The digital data optionally includes video data having an 
image and/or frame rate format selected from the common video formats listed in 
Table 1, for example, the digital data optionally has a 1280 pixel by 720 line 
format, a frame rate of 24 fps and a bit depth of approximately 24. As shown in 
Fig. 10, data are received by the computer 1004. For example, the aforementioned 
digital data (1280 pixel by 720 line format) are received by a computer (e.g., the 
computer 1004) having a PENTIUM® processor (Intel Corporation, Delaware) 
having a speed of 1.4 GHz (e.g., a PENTIUM® III processor). Consider another 
example wherein the digital data optionally has a 1920 pixel by 1080 line image 
format, a frame rate of 24 fps and a bit depth of approximately 24 bits. Data are 
received by a computer (e.g., the computer 1004) having two processors, wherein 
each processor has a speed of greater than 1.2 GHz, e.g., two AMD® processors 
(Advanced Micro Devices, Incorporated, Delaware). In general, a faster processor 
speed allows for a higher resolution image format and/or a higher frame rate. 
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Referring again to Fig. 10, after the computer 1004 has received the data, 
the data are transmitted to an input/output device 1008 capable of outputting data 
in a format. For example, one such I/O device is the FILMSTORE™ (Avica 
Technology Corporation, Santa Monica, Califomia) I/O device, which can output 
data according to the SMPTE 292M specification. The FILMSTORE™ I/O 
device is compression and encryption independent and has a DVD-ROM drive, six 
channel (5.1) digital audio output, and up to 15 TB or more of storage. The 
FILMSTORE™ I/O device stand-alone playback capability and, in a server 
configuration, can accommodate single or multi-screen playing, optionally with 
continuously changing storage, scheduling and distribution requirements. Suitable 
inputs to the FILMSTORE™ I/O device include, but are not limited to, satellite 
feeds, broadband connections and/or physical media. The I/O device 1008 is 
optionally a card located in the computer 1004 and, the computer 1004 may also 
output a VGA or other signal as described below. 

As shown in Fig. 10, output from the I/O device 1008 is transmitted to a 
monitor 1012 and/or a projector 1016. The monitor 1012 and/or the projector 
1016 optionally accept data in a format according to the SMPTE 292M 
specification. For example, the I/O device can transmit data to a LG2001™ 
projector (Lasergraphics Incorporated, Irvine, Califomia), which supports a variety 
of digital formats and digital input from a serial digital input, e.g., a 75 ohm BNC 
SMPTE 292M compliant signal cable. The LG2001™ projector can display 1920 
pixel by 1080 line resolution and also accept 16:9 high definition television signals 
in both 10801 and 720p formats. The LG2001™ digital projector also supports 
analog formats and inputs. Regarding monitors, consider the SyncMaster 240T 
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(Samsung Electronics Co., Ltd, South Korea), which can operate as a computer 
monitor as well as a widescreen DVD or HDTV display monitor. This display 
monitor offers both digital and analog inputs and supports a variety of image 
resolutions including 1920 pixel by 1200 line. Of course, the output from the I/O 
device 1008 may also feed a plurality of monitors and/or projectors. 

Overall, the exemplary method 1000 demonstrates delivery and playing of 
high resolution video (e.g., having a 1280 pixel by 720 line image format or a 
1920 pixel by 1080 line image format at, e.g., 24 fps). This exemplary method 
1000, for a 1280 pixel by 720 line image format and a 1920 pixel by 1080 line 
image format at 24 fps, provides 3-fold or 6-fold resolution increase, respectively, 
above standard definition DVD resolution at data rates below and/or equal to 
current DVD standard definition data rates. The application of compression to 
various formats allow content to be stored on standard definition DVDs and 
transmitted through existing standard definition pathways, such as, but not limited 
to, IP in digital TV transmissions and/or satellite direct broadcast. 

Regarding storage to a transportable storage medium, such as, but not 
limited to, a DVD disk, consider content having a 1920 pixel by 1080 line 
resolution, a frame rate of 24 fps and a color depth of 24 bits. Such content 
requires a bit rate of approximately 1.2 Gbps and two hours of content requires a 
file size of approximately 8.6 Tb. A compression ratio of approximately 250:1 
would reduce the file size to approximately 34 Gb, which would fit on a single 
sided DVD disk. Subjective and objective quality measures of such content are 
discussed in more detail below. 
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Various exemplary methods disclosed herein are capable of encoding 
(compressing) SMPTE 292 specification format data at a rate of 1.5 Gbps to a 
WINDOWS MEDIA™ format using WINDOWS MEDIA™ software, wherein 
the WINDOWS MEDIA™ format data are deliverable, optionally as a stream, at a 
rate of approximately 1.5 Mbps to approximately 10 Mbps or higher if desired. Of 
course, a hardware equivalent to the WINDOWS MEDIA™ software may be used 
as an alternative. 

Another exemplary method 1100 is shown in Fig. 11 wherein a computer 
1104 transmits data to a monitor 1112 and/or a projector 1116. In this exemplary 
method 1100, the computer 1104 receives data in an encoded format and/or 
converts data to a decompressed format. The computer 1104 also has software for 
decompressing (or decoding) data in a compressed (or encoded) format. After 
decompression (or decoding), video data are transmitted from the computer 1104 
to the monitor 1112 and/or the projector 1116. In general, the computer 1104 
contains appropriate hardware and/or software to support display of video data via 
the monitor 1112 and/or via the projector 1116. 

An exemplary monitor may support the video graphic array (VGA) and/or 
other display specifications or standards. In general, a VGA display system and 
other systems include sub-systems, such as, but not limited to, a graphics 
controller, display memory, a serializer, an attribute controller, a sequencer and a 
CRT controller. In the VGA display system, a computer CPU typically performs 
most of the work; however, a graphics controller can perform logical fimctions on 
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data being written to display memory. Display memory can be of any suitable 
size, for example, display memory may include a bank of 256k DRAM divided 
into 4 64k color planes. Further^ a VGA display system serializer receives display 
data from the display memory and converts it to a serial bit stream which is sent to 
an attribute controller. An attribute controller typically includes color tables, e.g., 
look up tables (LUTs) that are used to determine what color will be displayed for a 
given pixel value in display memory. A sequencer typically controls timings and 
enables/disables color planes. Finally, in a VGA display system, a CRT controller 
generates syncing and blanking signals to control the monitor display. 

Recently, new specifications have arisen that include, but are not limited to, 
super extended graphics array (SXGA) and ultra extended graphics array 
(UXGA). The SXGA specification is generally used in reference to screens with 
1280 x 1024 resolution; UXGA refers to a resolution of 1600 by 1200. The older 
specifications (VGA and SVGA) are often used simply in reference to their typical 
resolution capabilities. The Table 3, below, shows display modes and the 
resolution levels (in pixels horizontally by lines vertically) most commonly 
associated with each. 



Table 3. Exemplary video display system specifications 



System 


Pixel by Line Resolution 


VGA 


640 by 480 


SVGA 


800 by 600 


XGA 


1024 by 768 


SXGA 


1280 by 1024 
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UXGA 



1600 by 1200 



Some monitors support higher resolutions, for example, consider the 
SyncMaster 240T (Samsung Electronics Co., Ltd, South Korea), which can 
operate as a computer monitor as well as a widescreen DVD or HDTV display 
monitor. This display monitor offers both digital and analog inputs. Regarding 
projection, the exemplary method 1100 optionally uses a projector such as, but not 
limited to, the LG2001™ projector, which can display up to QXGA specification 
(e.g., 2048 pixel by 1536 line) resolution images directly from a computer. Of 
course, the output from the computer 1104 may also feed a plurality of monitors 
and/or projectors. 

Various exemplary methods, devices and/or systems described herein are 
also suitable for use with monitors having a lenticular lens or screen. Video 
suitable for viewing on such monitors is typically acquired through use of a 
plurality of cameras. Video data from each camera is generally spliced with that 
from other cameras and formatted to correspond to characteristics of a particular 
lenticular monitor. A commercially available exemplary monitor is marketed 
and/or sold under the mark SYNTHAGRAM™ (StereoGraphics Corporation, San 
Rafael, Califomia). Use of such a monitor with appropriate video data can 
provide a viewer with 'three-dimensional" perception. An exemplary method for 
handling video destined for display on a device that uses a lenticular lens 1200 is 
shown in Fig. 12. Images Ii through are taken at time "t" using, for example, 
"n" different cameras. In general, this process is repeated for a period of time, for 
example, "t" equals 0 seconds to "t" equals 100 seconds at a frame rate of 24 fps. 
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which would result in "100*24*n" total images. A slice is taken from each image, 
typically with respect to a reference time, to form a composite image It at a 
particular time "t". This process continues for the period of time. The resulting 
images It=o through It=ioo are then transmitted to a device 1220. The device 1220 is 
optionally a device having features of the device 610 described herein with 
reference to Fig. 6. Of course, the device may be a recorder or otiier device 
having features of devices described herein. According to this exemplary method 
1200, the composite images are structured, stored and encoded (e.g., compressed) 
into a stream or file(s) having, for example, an encoded format (e.g., WINDOWS 
MEDIA™ format). The resulting video stream and/or file(s) are then transmitted 
to a computer having a decoder and player 1230 which allows for display of the 
video onto a lenticular display 1240. 

An exemplary method for displaying images from film 1300 is shown in 
Fig. 13. In a conversion block 1304, fihn images are converted to a digital data 
stream and/or file(s). Next, in an encoding block 1308, the digital data stream 
and/or file(s) are encoded to a format suitable for a stream(s) and/or a file(s). 
Following the encoding block 1308, in a decoding block 1312, the stream(s) 
and/or file(s) are decoded to data in a digital and/or an analog video format 
suitable for display. Following the decoding block 1312, the data in a digital 
and/or an analog format are displayed. 

According to the exemplary method 1300, fihn rniages (or frames) are 
optionally converted to digital data with an image format wherein one of the pixel 
or line sizes is at least 720 and the other pixel or line size is greater than 576. The 
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digital data are then optionally converted to a format for storage and then 
optionally encoded to a WINDOWS MEDIA™ format stream and/or file using 
encoding software, such as, but not limited to, aforementioned encoding software 
that uses a video codec. The encoded format stream and/or file is then locally 
and/or remotely decoded (e.g., using a suitable video codec) and optionally 
transmitted to a display device (e.g., a monitor, a projector, etc.) wherein the 
decoded video images are displayed with an image format wherein one of the 
pixel or Ime sizes is at least 720 and Ihe other pixel or line size is greater than 576. 
In the case that the encoded format stream and/or file is transmitted and/or stored, 
decoding of the stream and/or file optionally includes padding (e.g., zero padding). 
Further, the encoded format stream and/or file optionally contain variable-bit-rate 
information. 

Fig. 14 is a graph of bit rate in Gbps (ordinate, y-axis) versus processor 
speed for a computer having a single processor (abscissa, x-axis). The graph 
shows data for encoding video and for decoding video. Note that the data points 
lay along approxunately straight lines in the x-y plane (a solid line is shown for 
decoding and a dashed line is shown for encodmg). A regression analysis shows 
that decoding has a slope of approximately 0.4 Gbps per GHz processor speed and 
that encoding has a slope of approximately 0.1 Gbps per GHz processor speed. In 
this particular graph, it is apparent that, with reference to the foregoing discussion, 
that resolution, fi^e rate and color space need not adhere to any specific format 
and/or specification. The ordinate data was calculated by multiplying a pixel 
resolution number by a line resolution number to arrive at the number of pixels per 
frame and then multiplying the pixels per frame number by a frame rate and the 
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number of color information bits per pixel. Thus, according to various exemplary 
methods, devices and/or systems described herein, encoding and/or decoding 
performance characteristics, if plotted in a similar manner would produce data 
lying approximately along the respective lines as shown in Fig. 14. Thus, 
according to various aspects of exemplary methods, devices and/or systems 
described herein, a computer having an approximately 1.5 GHz processor has can 
decode encoded video at a rate of approximately 0.6 Gbps, e.g., 1.5 GHz 
multiplied by 0.4 Gbps/GHz, and therefore, handle video having a display rate of 
approximately 0.5 Gbps, e.g., video having a resolution of 1280 pixel by 720 line, 
a frame rate of 24 frames per second and a color bit depth of 24 bits. Note that for 
decoding, the rate is given based on a video display format and not on the rate of 
data into the decoder. 

Various exemplary methods, devices, systems, and/or storage media 
discussed herein are capable of providing quality equal to or better than that 
provided by MPEG-2, whether for DTV, computers, DVDs, networks, etc. One 
measure of quality is resolution. Regarding MPEG-2 technology, most uses are 
limited to 720 pixel by 480 line (345,600 pixels) or 720 pixel by 576 line (414,720 
pixels) resolution. In addition, DVD uses are generally limited to approximately 
640 pixel by 480 line (307,200 pixels). Thus, any technology that can handle a 
higher resolution will inherently have a higher quality. Accordingly, various 
exemplary methods, devices, systems, and/or storage media discussed herein are 
capable of handling a pixel resolution greater than 720 pixels and/or a line 
resolution greater than approxhnately 576 lines. For example, a 1280 pixel by 720 
line resolution has 921,600 pixels, which represents over double the number of 
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pixels of the 720 pixel by 576 line resolution. When compared to 640 pixel by 
480 line, the increase is approximately 3-fold. On this basis, various exemplary 
methods, devices, systems, and/or storage media achieve better video quality than 
MPEG-2-based methods, devices, systems and/or storage media. 

Another quality measure involves measurement of peak signal to noise 
ratio, known as PSNR, which compares quality after compression/decompression 
with original quality. The MPEG-2 standard (e.g., MPEG-2 Test Model 5) has 
been thoroughly tested, typically as PSNR versus bit rate for a variety of video. 
For example, the MPEG-2 standard has been tested using the "Mobile and 
Calendar" reference video (ITU-R library), which is characterized as having 
random motion of objects, slow motion, sharp moving details. In a CCIR 601 
format, for MPEG-2, a PSNR of approximately 30 dB results for a bit rate of 
approximately 5 Mbps and a PSNR of approximately 27.5 dB for a bit rate of 
approximately 3 Mbps. Various exemplary methods, devices, systems, and/or 
storage media are capable of PSNRs higher than those of MPEG-2 given the same 
bit rate and same test data. 

Yet another measure of quality is comparison to VHS quality and DVD 
quality. Various exemplary methods, devices, systems, and/or storage media are 
capable of achieving DVD quality for 640 pixel by 480 line resolution at bit rates 
of 500 kbps to 1.5 Mbps. To achieve a 500 kbps bit rate, a compression ratio of 
approximately 350:1 is required for a color depth of 24 bits and a compression 
ration of approximately 250:1 is required for a color depth of 16 bits. To achieve a 
1.5 Mbps bit rate, a compression ratio of approximately 120:1 is required for a 
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color depth of 24 bits and a compression ratio of approximately 80:1 is required 
for a color depth of 16 bits. Where compression ratios appear, one would 
understand that a decompression ratio may be represented as the reverse ratio. 

Yet another measure of performance relates to data rate. For example, 
while a 2 Mbps bit rate-based "sweet spot" was given in the background section 
(for a resolution of 352 pixel by 480 line), MPEG-2 is not especially usefiil at data 
rates below approximately 4 Mbps. For most content a data rate below 
approximately 4 Mbps typically corresponds to a high compression ratio, which 
explains why MPEG-2 is typically used at rates greater than approximately 4 
Mbps (to approximately 30 Mbps) when resolution exceeds, for example, 352 
pixel by 480 line. Thus, for a given data rate, various exemplary methods, 
devices, systems, and/or storage media are capable of delivering higher quality 
video. Higher quality may correspond to higher resolution, higher PSNR, and/or 
other measures. 

Various exemplary methods, devices, systems and/or storage media are 
optionally suitable for use with games. While the description herein generally 
refers to "video" many formats discussed herein also support audio. Thus, where 
appropriate, it is understood that audio may accompany video. Although some 
exemplary methods, devices and exemplary systems have been illustrated in the 
accompanying Drawings and described in the foregoing Detailed Description, it 
will be understood that the methods and systems are not limited to the exemplary 
embodiments disclosed, but are capable of numerous rearrangements. 
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modifications and substitutions without departing from the spirit set forth and 
defined by the following claims. 
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